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(U)'  Display  Size  and  Target  Acquisition  Performance,  by  Michael 
J.  Bames.l  China  Lake,  Calif.,  Naval  Weapons  Center,  January  1978.  44 
pp.  (NWCkp  6006,  pubUcation  UNCLASSIFIED.) 

(U)^wo  experiments  were  conducted  to  find  factors  that  have 
an  important  effect  on  display  size  criteria  in  a cockpit -display  system. 
Subjects  in  both  experiments  detected  military  targets  simulating  images 
from  a TV  camera  looking  obliquely  forward  as  it  is  flown  over  the 
terrain.  ^ 

The  results  of  the  two  experiments  indicated  that  the 
physical  size  of  a television  monitor  is  not  an  important  factor  if  MTF 
and  visual  angle  are  held  constant.  Of  the  factors  studied,  the  four 
most  important  were:  number  of  targets,  visual  angle  of  targets,  target 
contrast,  and  target  configuration. 

ii^The  data  from  the  second  experiment  were  used  to  generate 
a multiple  regression  model.  The  relationship  between  target  visual  angle 
and  display  size  allowed  the  regression  model  to  be  used  to  predict 
performance  as^  function  of  display  size. 
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INTRODUCTION 


The  design  and  layout  of  an  aircraft  cockpit  requires  consideration 
of  the  space  limitations,  display  and  control  technology,  the  mission  of 
the  aircraft,  and  the  associated  tasks  that  the  aircrew  must  perform. 
Current  Navy  aircraft  use  cathode  ray  tubes  (CRTs)  to  display  symbology, 
as  well  as  real-world  Imagery  from  television  (TV)  or  forward-looking 
Infrared  systems  (FLIRs) . The  A-7E  aircraft  has  a CRT  display  for 
Imagery  from  TV-gulded  weapons , and  the  A-7E  FLIR  system  uses  the  CRT 
In  the  head-up  display  as  Its  monitor.  The  A-6E  aircraft  also  uses 
CRTs,  and  the  F-18  now  under  development  uses  three  CRTs  for  alphanumeric, 
symbolic,  and  FLIR  displays. 

Whenever  the  tasks  of  display  design,  cockpit  layout,  or  air-to- 
ground  target  acquisition  with  electro-optical  sensors  arise,  the 
question  of  display  size  also  comes  up.  The  experiments  described  In 
this  report  were  conducted  to  provide  some  quantitative  answers  to 
this  question.  The  experiments  provide  results  that  relate  the  size  of 
a display  to  the  operator's  performance  at  target  acquisition.  The 
results  can  be  used  to  specify  system  requirements , or  to  predict 
target  acquisition  performance  with  existing  systems. 


OVERVIEW 

Two  laboratory  experiments  were  conducted  In  which  subjects  were 
required  to  search  a CRT  display  for  ml lit ary- type  targets;  under  one 
condition  they  also  simultaneously  monitored  another  display  for  "mal- 
function" Indications.  The  display  was  generated  by  videotaping  the 
Imagery  from  a realistic  terrain  model  and  then  playing  It  back  to  the 
subjects.  The  targets  were  tanks,  missiles,  houses,  and  trucks. 

The  Imagery  was  similar  to  that  produced  by  a TV  camera  looking 
forward  and  obliquely  downward  as  It  Is  flown  over  the  terrain. 

In  the  first  experiment,  eleven  factors  Identified  In  a literature 
review  were  examined  using  a screening  (partial  factorial)  design.  The 
four  most  Important  factors,  accounting  for  50Z  of  the  variance,  were 
then  used  as  the  basis  for  the  second  experiment  which  used  a full 
factorial  design. 

Both  an  analysis  of  variance  and  a regression  analysis  were  performed 
on  the  data.  A model  was  generated  from  the  data  which  predicted  perform- 
ance (percentage  correct)  for  different  display  sizes  under  a variety 
of  different  conditions.  Subject  differences  were  Investigated  using 
statistics  developed  from  signal  detection  theory. 
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GENERAL  EXPERIMENTAL  APPROACH 

The  relationship  of  display  size  and  human  performance  is  a complex 
issue  when  considered  in  a systems  context.  For  example,  an  attack 
aircraft  has  many  system  parameters  that  affect  pilot  performance.  When 
these  vary,  the  size  of  the  display  required  also  varies.  Therefore, 
the  effect  of  display  size  on  pilot  performance  should  be  measured  as 
part  of  a general  empirical  model  predicting  operator  performance  for  a 
cockpit- display  system.  The  problem  with  generating  such  a model  is 
that  the  number  of  factors  that  must  be  considered  is  too  large  to  use 
traditional  experimental  designs. 

The  approach  used  for  this  study  involves  a multivariate  experi- 
mental program. ^ The  result  of  such  an  approach  is  a predictive  model 
relating  operator  performance  to  a selected  subset  of  system  parameters. 

The  multivariate  approach  involves  more  than  simply  varying  a 
number  of  arbitrarily  chosen  variables  in  the  same  experiment.  Rather, 
it  is  a multistage  experimental  program  consisting  of  three  major  steps: 
(1)  review  of  the  literature  to  identify  potentially  important  factors 
in  the  system  being  investigated;  (2)  choice  of  10  or  more  of  these 
factors  to  be  used  in  a statistical  screening  experiment;  and  (3)  the 
use  of  traditional  experimental  designs^  and  multiple  regression 
techniques^  to  generate  a mathematical  model  to  predict  system  perform- 
ance. At  each  stage,  the  number  of  factors  considered  is  reduced  until 
systems  performance  is  described  in  terms  of  a small  number  of  important 
factors . 


* Hughes  Aircraft  Co.  luononiical  Multifactor  Designs  for  Human  Factors  Fnginecring 
Fxpertments,  by  C.  W.  Simon.  Culver  City.  CA,  HAC,  June  1973.  (Technical  Report  No.  P7.3-326, 
publication  UNCLASSIFIED.) 

^ Roger  E.  Kirk.  Experimental  Design:  Procedures  for  the  Behavioral  Sciences.  Belmont,  CA. 
Brooks-Cole,  1968,  pp.  171-244. 


! 

1 


1966. 


N.  R.  Draper  and  H.  Smith.  Applied  Regression  Anaivsis.  New  York,  John  Wiley  & Sons. 
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EXPERIMENT  1 


OVERVIEW 

The  purpose  of  this  experiment  was  to  identify  factors  which  most 
Influenced  pilot  performance  for  a detection  task.  The  Independent  i 

variables,  their  high  and  low  level  values,  and  the  fixed  parameters 
of  the  experiment  are  given  in  Ta’'le  1. 


TABLE  1.  Experiment  1,  Fixed  Parameters  and  Independent  Variables. 


Fixed 

, parameters 

Value 

Independent 

variables 

High  level 

Low  level 

Terrain  scale 

800:1 

Display  size 

9-inch 

2-inch 

Altitude 

^ Range  from 

3,200  fta 

Visual  angle 
of  target 

22  min 

11  min 

2.6  nml^ 

Simulated  air- 

259  knots 

460  knots 

sensor  to  ground 
at  center  of  FOV 

speed 

Field  of  view 

2* 

Mode  of  image 

Moving  or 

Series  of 

presentation 

snow  plow 

stills 

Footprint 

925  m 

Operator 

1 bit 

2 bits 

length 

uncertainty 

Footprint 

305  m 

Approx,  target- 

-0.48 

-0.27 

width 

background 

contrast 

Terrain 

(European) 

Signal-to- 

28  dB 

12  dB 

clutter 

medium 

noise  ratio 

Targets 

Tank, 

Television 

300  lines 

175  lines 

truck 

resolution 

House, 

Task  Load 

One  display 

Three  displays 

missile 

No.  of  targets 

Six 

Three 

Sub  j ects 

• * • 

. . . 

^ Feet  and  nautical  miles,  rather  than  metric  terms,  are  most 
generally  used  for  altitude  and  range,  respectively. 
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APPARATUS 

Four  525-line  TV  monitors  were  used  as  displays.  Either  a 2-inch 
3Q  monitor  or  a 9-inch  Conrac  monitor  matched  for  modulation  transfer 
function  (MTF)  (Appendix  A)  was  centered  in  a black  display  console. 

On  either  side  of  the  center  monitor  were  two  Shibaden  9-inch  monitors. 
Three  Ampex  videotape  recorders  (VTRs)  provided  input  for  the  monitors. 
The  output  cable  from  the  VTR  to  the  center  monitor  was  routed  through 
a noise  mixer.  Noise  from  a General  Radio  random  noise  generator  was  fed 
into  an  SRL  model  262  low-pass  filter  set  at  6 MHz.  The  output  from  the 
filter  was  mixed  with  the  video  signal  from  the  VTR  and  used  as  the  input 
for  the  center  TV  monitor  (Figure  1) . 

A stationary  Sony  video  camera  observing  a movable  terrain  model 
having  different  medium-clutter  scenes  provided  imagery  for  the  center 
monitor.  One  of  four  targets  (missiles,  trucks,  tanks,  and  houses), 
each  1 cm  across,  was  located  on  the  terrain  model  during  target  trials. 
Figure  2 shows  a typical  scene  on  the  center  TV  monitor. 

A headrest  was  used  to  maintain  a constant  distance  between  the  sub- 
ject and  displays  during  the  experiment.  The  subject  heard  the  response 
tone  through  earphones  and  used  a two-button  response  box.  Signals  from 
the  box  were  fed  into  a Honeywell  vislcorder  to  record  responses. 


SUBJECTS 

Six  engineers  employed  at  the  Naval  Weapons  Center  were  used  as 
subjects.  The  subjects  ranged  in  age  from  30  to  37  years.  All  subjects 
tested  at  least  20/20  (corrected)  on  a Bausch  and  Lomb  Armed  Forces 
Vision  Tester  for  both  far  and  near  binocular  visual  acuity. 


METHODOLOGY 

Design 


The  screening  experiment  used  a saturated  fractional-factorial 
design.  Computational  procedures,  advantages  and  limitations  of  such 
a design  are  discussed  in  detail  by  Simon. ^ Two  orthogonal  blocks  of 
16  conditions  each  (l.e.,  basic  and  foldover  blocks^)  were  used  to 
estimate  the  effects  of  10  factors  and  21  strings  of  second-order 
interactions . 
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figure  1.  Apparatus  and  Experimental  Setup. 
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The  32  conditions  were  within-sub ject  conditions.  Between-s object 
estimates  of  trend  effects  and  subject  differences  were  made  using  two 
groups  with  three  subjects  each.  Each  of  the  32  experimental  conditions 
constituted  an  experimental  run.  Furthermore,  there  were  eight  trials 
per  run  making  256  trials  per  subject.  One  orthogonal  block  of  runs  was 
given  in  the  first  half  of  the  experimental  session  and  the  other  block 
was  given  in  the  second  half.  One  group  of  subjects  saw  the  conditions 
(runs)  within  the  blocks  in  the  orders  1-16  and  17-32.  The  other  group 
of  subjects  saw  the  conditions  in  the  reverse  order  16-1  and  32-17. 

Each  subject  had  a practice  session  in  the  morning  lasting  one  hour 
and  consisting  of  192  practice  trials  with  feedback.  The  experimental 
session  took  place  in  the  afternoon  and  lasted  for  an  hour  and  a half. 
Experimental  instructions  were  given  to  the  subjects  in  the  morning 
session. 


FIGURE  2.  Typical  Terrain  Scene. 
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Procedure 


The  subject  sat  a premeasured  distance  from  the  center  monitor  with 
both  thumbs  on  the  response  box.  The  run  started  with  the  image  of  a 
number  on  the  central  monitor  being  replaced  by  a terrain  scene.  The 
run  was  divided  into  eight  segments  (trials)  showing  different  portions 
of  a continuous  stretch  of  terrain  simulating  4 nmi.  A trial  encompassed 
one  terrain  segment.  After  each  trial,  a tone  sounded,  indicating 
the  start  of  the  next  trial.  The  tone  also  signalled  the  subject  to 
make  a forced  choice  by  pressing  the  YES  button  if  he  felt  that  there 
was  a target  in  the  scene,  and  pressing  the  NO  button  if  he  felt  there 
was  no  target  in  the  scene.  The  forced  choice  method  (i.e.,  YES-NO 
response)  was  used  in  order  to  take  advantage  of  statistics  developed 
in  signal  detection  theory.  Using  these  statistics,  it  is  possible  to 
measure  the  subjects’  tendency  to  miss  targets  rather  than  report 
doubtful  (but  possible)  targets,  as  well  as  his  ability  to  detect 
targets . 

The  same  terrain  was  viewed  for  each  run.  Within  each  run  targets 
were  placed  in  the  middle,  left,  or  right  sections  of  the  terrain  equally 
often.  However,  the  placement  of  targets  on  any  trial  was  random  Fa'. h 
trial  had  a 0.50  probability  of  containing  a target.  The  targets  were 
arranged  in  a column  situated  horizontally  on  the  monitor. 


Independent  Variables 

Display  size  was  varied  by  using  either  a 2-  or  a 9-inch  TV  monitor. 
Four  viewing  distances  (30,  61,  91,  or  183  cm)  were  used  so  that  visual 
angle  (measured  horizontally)  could  be  varied  independently  of  display 
size. 


Simulated  airspeed  was  calculated  for  the  time  it  would  take  an 
airplane  to  transverse  one  field  of  view  (FOV) , given  the  fixed  param- 
eters in  Table  1.  Two  methods  of  airspeed  simulation  were  used,  depending 
on  the  mode  of  image  presentation  condition.  For  the  "snow  plow"  or 
"moving"  mode,  the  terrain  model  was  moved  at  a rate  of  1 FOV/4  sec  and 
1 FOV/7  sec,  simulating  airspeeds  of  453  and  259  knots,  respectively. 

In  the  "series  of  stills"  mode,  each  scene  segment  was  static,  but  was 
replaced  immediately  at  intervals  of  4 or  7 seconds  by  the  next  scene 
segment.  The  tones  signalling  the  subject  to  make  a forced  choice  were 
coincident  with  a change  of  FOV  for  both  modes  of  presentation. 

The  uncertainty  (U)  of  a particular  target  being  present  was 
measured  according  to  Equation  1 in  bits. 

U = log2  s (1) 
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where  s represents  the  number  of  possible  targets.  Thus,  if  the  subjects 
were  told  that  any  of  four  targets  were  possible  for  a particular  run, 
the  uncertainty  would  be  2 bits. 

"Number  of  targets"  refers  to  the  number  of  targets  actually  present 
in  a target  configuration.  Each  configuration  contained  only  one  type 
of  target.  For  example,  six  targets  would  be  a column  of  six  tanks  or 
six  trucks,  etc. 

Because  the  luminance  of  the  terrain  background  varied  a great 
deal,  the  terrain  was  randomly  sampled  as  to  luminance  level  in  the 
following  manner.  The  four  terrain  segments  used  in  the  experiment  were 
each  sampled  at  eight  random  locations  and  the  luminances  for  these  32 
locations  were  averaged  to  give  an  average  background  luminance  (BL) . 

The  top  and  bottom  of  the  four  targets  were  each  measured  with  a photom- 
eter in  the  middle  of  the  four  terrain  scenes  and  these  32  luminance 
values  were  used  to  find  an  average  target  luminance  (TL) . This  was 
done  for  both  the  light  and  dark  targets.  The  lighter  targets  were 
painted  with  flat  earth  Pactra  enamel  (XF-52) . The  darker  targets  were 
painted  with  Pactra  flat  dark  olive  (XF-51)  mixed  with  flat  black 
enamel.  The  following  equation  was  used  to  compute  this  average 
contrast . 

Contrast  = (2) 


Both  types  of  target  were  darker  than  their  backgrounds  (i.e.,  negative 
contrast)  and  are  referred  to  as  light  and  dark  only  to  denote  the 
relative  appearance  of  the  targets. 

The  camera  was  defocused  electronically  and  the  resolution  measured 
for  both  the  focused  and  defocused  conditions  using  a RETMA  chart. 

Noise  was  mixed  with  the  VTR  signal  and  measured  with  an  oscillo- 
scope at  the  output  of  the  mixer'*  to  determine  slgnal-to-noise  (S/N) 
ratio.  Equation  3 was  used  to  compute  the  ratio  in  decibels. 


S/N  ratio  = 20  log 


(peak-peak  signal) 
^displayed  noise j 


(3) 


Franklin  and  T.  Hatley.  “Don’t  Eyeball  Noise,”  tUecinmic  Design,  22  November  l‘)73, 
pp.  184-187. 
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Task  loading  was  accomplished  by  having  the  subjects  monitor  two 
additional  TV  displays.  The  displays  each  presented  five  gauges 
showing  possible  engine  malfunctions  (average  rate:  one  malfunction 
per  22  sec).  Further  details  on  the  malfunction  displays  are  given  by 
Wagner^  (all  monitors  In  the  present  experiment  were  black-and-white) . 
The  subject  was  told  to  report  verbally  when  one  of  the  Indicators  on 
the  gauges  was  out  of  tolerance.  This  Information  wac  recorded  and 
later  compared  with  the  taped  malfunctions. 

Dependent  Variable 


The  Independent  measure  was  the  number  of  correct  decisions.  The 
percentage  of  correct  decisions  was  corrected  for  chance  by  subtracting 
the  number  wro'.g  from  the  number  correct  before  computing  the  proportion 
correct. ^ 


RESULTS  AND  DISCUSSION 

Although  the  subjects  reported  that  the  task  was  difficult,  they 
performed  at  an  average  rate  of  69%  correct  for  the  detection  task  over 
all  conditions.  For  the  malfunction  reports,  all  subjects  performed 
at  nearly  100%  and  therefore  no  further  analysis  was  done  on  these  data. 

The  effect  of  the  factors  was  determined  by  the  percentage  of  the 
experimental  variance  each  accounted  for  (l.e. , sum  of  squares  (ss)  for 
each  factor  v total  sum  of  squares) . This  measure  Is  straightforward 
and  does  not  require  the  assumptions  that  formal  testing  methods  do. 
Furthermore,  It  Is  a measure  of  the  relative  Importance  of  the  factors, 
making  It  a particularly  appropriate  measure  for  a screening  experiment 
wherein  the  main  purpose  is  to  choose  a subset  of  the  most  important 
variables  for  further  experimentation. 

Table  2 gives  a breakdown  of  the  sum  of  squares  and  percent  of 
variance  for  the  main  factors  and  second-order  Interactions.  Block 
effects  are  a measure  of  within-session  learning  (first-half  performance 
as  opposed  to  second-half  performance) . Since  the  subjects  were  run  in 
two  groups  depending  on  the  order  in  which  they  observed  the  experimental 
conditions,  the  subject  effects  were  partitioned  into  subject  differences 
(A)  and  trend  effects  (B) . 


^ Naval  Weapons  Center.  I•'xp^•rwlclus  With  Color  Coding  on  Television,  by  Dan  W.  Wagner. 
China  Lake,  CA,  NWC.  January  1977.  (NWC  TP  5952,  publication  UNCLASSIFIED.) 

W.  Lee.  Decision  Theory  and  Hinnon  liehuvior.  New  York,  John  Wiley  and  Sons,  1971. 


11 


1 


NWC  TP  6006 


TABLE  2.  Effect  of  Main  Factors  and  Strings  of  Interactions 
for  the  Partial  Factorial  Experiment. 


No.  of  targets  (T) 
Subjects 

(A)  Trends 

(B)  Differences 
Visual  angle  (V) 
Signal-to-noise  ratio  (S) 
Contrast  (C) 

No.  of  tasks  (N) 

Blocks 

Resolution  (R) 

Display  size  (D) 

Airspeed  (A) 

Presentation  mode  (M) 
Uncertainty  (U) 

D*V 

D*S 

D*M 

V*S 

V*M 

U*C  + T*S 

U-S  + U*T 

C*R  + M-T 

U*N  + A*T 

C*S  + M*R  + A*N  + C*T 

M*S  + C*R  + V*N 

A*S  + V*R  + C*N 

M*C  + R‘S  + D‘N 

VC  + D'R  + S‘N 

VS  + D'S  + N*R  + A*M 

V*A  + M*U  + R*T 

A‘U  + T*N 

D*M  + V*U  + D*T 

D‘U  + V*T 

D*C  + V*S  + A'R  + M*N 


Percent 

1 

variance 

23 

^ i 
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Figure  3 shows  the  relative  importance  of  the  four  most  important 
factors  compared  with  the  other  factors  and  second-order  interactions. 
These  four  factors  (i.e.,  number  of  targets,  visual  angle,  S/N  ratio, 
and  contrast)  accounted  for  50%  of  the  experimental  variance  and  were 
chosen  for  further  experimentation.  Subject  effects  also  accounted  for 
a significant  portion  of  the  variance.  Subject  differences  were 
analyzed  using  statistics  developed  from  signal  detection  theory  and 
are  discussed  with  the  results  from  the  second  experiment. 


CUMULATIVE 

PERCENT 

VARIANCE  FACTOR 


NO.  OF  TARGETS 

SUBJECTS 

TARGET'S  VISUAL  ANGLE 

SIGNAL-TO-NOISE  RATIO 
CONTRAST 


ALL  OTHER  FACTORS  AND 
SECOND-ORDER  INTERACTION 
STRINGS 


FIGURE  3.  Variance  Accounted  for  by 
Experimental  Variables. 


It  should  be  noted  that,  because  of  the  unavoidable  confounding 
involved  in  using  a partial  factorial  design.  Figure  3 should  be  thought 
of  as  a guide  to  further  research  rather  than  an  absolute  measure  of  the 
magnitudes  of  the  various  factors  used  in  the  screening  experiment . ^ ^ 
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Perhaps  Che  most  Important  result  of  the  screening  experiment  was 
that  the  size  of  the  display,  Independent  of  the  visual  angle  of  the 
target,  was  a relatively  unimportant  factor.  Most  previous  studies  have 
varied  Che  visual  angle  of  the  target  scene  at  the  same  time  they  varied 
display  size.’’®  Therefore,  It  Is  difficult  to  tell  whether  their  results 
are  due  to  the  visual  angle  of  the  target  scene  (or  targets)  or  the 
actual  size  of  the  display.  Moreover,  there  Is  some  experimental 
evidence  that  performance  Is  Improved  by  viewing  a large  display  at  a 
greater  distance  than  a small  display  while  keeping  a constant  visual 
angle.® The  present  study  and  research  by  Bruns Indicate  any  such 
Improvement  Is  negligible  when  considered  In  terms  of  the  total  cockpit- 
display  system.  Because  of  these  results.  It  was  decided  that  visual 
angle  and  not  the  physical  size  of  the  display  would  be  the  parameter 
of  interest  for  the  second  experiment. 

Also,  Che  visual  angle  of  the  target  Is  a more  general  factor,  since 
It  Is  related  to  changes  In  display  size,  FOV,  and  range.  The  relation- 
ship between  these  factors  and  target  visual  angle  can  be  used  to  predict 
performance  If  Che  relationship  between  visual  angle  and  performance  Is 
known.  The  functional  relationship  between  visual  angle  and  display 
size  Is  derived  In  the  "Applications"  section  (p.  23).  Given  that  FOV 
and  range  are  fixed  parameters,  performance  can  then  be  predicted  In 
terms  of  display  size. 


i 


1 


^ D.  W.  Craig  and  M.  L.  Hershberger.  “Synthetic  Aperture  Radar  Tactical  Target  Acquisition 
Research,”  in  Proceedings  of  the  Human  Factors  Society.  21st  Annual  Meeting,  1977,  pp. 
244-246. 

® Rome  Air  Development  Center.  Real-Time  Display  Parameters  Study,  by  Marjorie  J.  Krebs 
and  Carl  P.  Graf.  Griffis  Air  Force  Base,  N.Y.,  RADC,  September  1973.  (TR-73-300,  publication 
UNCLASSIFIED.) 

^ Naval  Training  Device  Center.  Parameters  for  Simulation  of  the  Pilot’s  Real  World,  by 
Halim  Ozkaptan.  Orlando,  FL,  NTDC,  1969.  (NAVTRADEVCEN  Report  68-C-0153-1,  publication 
UNCLASSIFIED.) 

A.  Chapanis  and  L.  C.  Scarpa.  “Readability  of  Dials  at  Different  Distances  with  Constant 
Visual  Angle,"  in  HUM.  Factors,  Vol.  9 (1967),  pp.  38—43. 

* * Pacific  Missile  Test  Center.  Dynamic  Target  Identification  on  Television  as  a Function  of 
Displav  Size,  Fiewing  Distance  and  Target  Motion  Rate,  by  R.  A.  Bruns.  Port  Mugu,  CA,  PMTC, 
1970.  (TP-70-60,  publication  UNCLASSIFIED.) 
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EXPERIMENT  2 


OVERVIEW 

Five  factors  were  examined  using  a 3x3x3x2x2  full  factorial  experi- 
mental design  (Table  3) . The  data  were  used  to  generate  a regression 
model  which  predicted  performance  under  different  levels  of  visual  angle 
(or  display  size),  number  of  targets,  target  configuration,  and  contrast 
levels.  Applications  of  the  regression  model  and  a correction  for  the 
effects  of  vibration  are  suggested. 

Finally,  the  results  of  the  two  experiments  are  discussed  with 
emphasis  on  the  appropriateness  of  the  model  and  a signal  detection 
analysis  of  subject  differences. 


TABLE  3.  Experiment  2,  Fixed  Parameters  and  Independent  Variables. 


Fixed 

parameters 

Value 

Independent  variables 

Level 

Value 

Terrain  scale 

800:1 

Visual  angle  of  the 

1 

7 min 

target 

2 

27  min 

Altitude 

3,200  ft^ 

47  mi.n 

3 

Range 

2.6  nmi^ 

Horizontal  field 

2® 

Number  of  targets 

1 

One 

of  view 

2 

Four 

Footprint  length 

925  m 

3 

Seven 

Footprint  width 

305  m 

Signal-to-noise  ratio 

1 

28  dB 

Terrain  clutter 

(European) 

medium 

2 

17  dB 

Targets 

Tank,  truck 

3 

13  dB 

Mode  of 

Series  of 

Contrast 

1 

Light 

presentation 

stills 

(-27%) 

Television 

resolution 

Task  load 

300  lines 

One 

Configuration 

2 

Dark 

(-48%) 

1 

Linear 

Airspeed 

363  knots 

2 

Random 

Task 

Detection 

Dependent  Variable 

Percent  Correct 

3 Feet  and  nautical  miles,  rather  than  metric  terms,  are  most 
generally  used  for  altitude  and  range,  respectively. 
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APPARATUS  ] 

1 

I 

The  same  basic  apparatus  was  used  for  the  second  experiment  except  - , ^ 

that  there  was  only  one  TV  monitor  (Shlbaden,  9-inch  diagonal)  and  a j 

single  VTR  Input.  ! 

I 

SUBJECTS  j j 

Four  of  the  subjects  uscU  In  the  first  experiment,  plus  two  addi-  j j 

tlonal  subjects,  participated  In  the  second  experiment.  The  two  i | 

I additional  subjects  had  been  In  previous  human  factors  experiments  | i 

' involving  the  terrain  model  and  had  20/20  corrected  binocular  vision.  | j 

■ ’ i I 

METHODOLOGY  j 

j 1 

Design  j 

i 1 

' The  design  was  a complete  factorial  wi thin-subject  design.  Table  4 ! 

^ shows  the  within-subject  partial  counterbalancing  scheme  for  one  subject.  j 

Each  cell  was  replicated  four  times  for  a total  of  432  data  points  per  j j 

subject.  Signal-to-noise  ratio  and  visual  angle  were  completely  counter-  j | 

balanced  between  subjects.  The  subjects  were  given  a rest  period  every  . j 

48  trials  and  a longer  rest  period  every  144  trials.  i j 

I i 

A morning  practice  session  consisting  of  300  practice  trials  (with  I 

feedback)  and  a 1-hour  experimental  session  the  same  afternoon  were  | | 

held  for  each  subject.  i 

i 

i 

Procedure 

i 

The  same  basic  procedure  was  used  as  in  the  first  experiment.  The  | | 

number  of  trials  per  run  was  increased  to  16,  four  of  which  were  of  the  I j 

same  experimental  condition.  The  run  card  starting  each  run  had  the  ! | 

number  of  targets  possible  printed  on  it  for  that  run.  The  durations  ' ] 

of  the  trial  and  the  interval  between  runs  were  both  5 seconds . Target  i 

placement,  type  of  target,  and  terrain  scenes  were  counterbalanced  within 

experimental  conditions.  | 

■ j 

Independent  Variables 

The  values  of  the  fixed  parameters  that  were  independent  variables 
in  the  first  experiment  are  presented  in  Table  3.  The  values  listed  in  ' 

Table  3 for  visual  angle,  number  of  targets,  S/N  ratio,  and  contrast 

were  computed  in  the  same  manner  as  in  the  first  experiment.  The  viewing  I 

i 

I 
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distances  used  to  compute  the  visual  angle  were  36,  64,  and  247  cm. 
"Configuration"  referred  to  the  way  In  which  the  targets  were  aligned. 
A linear  configuration  meant  that  the  targets  were  aligned  In  a column 
two  target  lengths  apart  and  lying  along  a horizontal  line  (on  the  TV 
monitor) . For  the  random  configuration  condition,  the  targets  were 
randomly  arranged  with  adjacent  targets  being  two  target  lengths  apart. 


Dependent  Variable 

The  same  dependent  variable  was  used  In  both  experiments.  In 
addition,  d'  and  3,  measures  from  signal  detection  theory,  were  used  to 
compare  subject  differences. 


RESULTS  AND  DISCUSSION 
Analysis  of  Variance 

Table  5 summarizes  the  analysis  of  variance  (ANOVA)  performed  on 
the  data  for  the  second  experiment.  (The  full  ANOVA  table  Is  given  In 
Appendix  B.) 


TABLE  5.  Experiment  2,  Summary  of  ANOVA. 


Factors 

F-score 

Significance  level 

Number  of  targets 

47 

P < .01 

Visual  angle 

40 

P < .01 

Configuration 

36 

P < .01 

Contrast 

22 

P < .01 

Signal- to-nolse  ratio 

3 

NS 

Configuration  by  contrast 

11 

P < .05 

Contrast  by  number 

8 

P < .01 

Configuration  by  contrast 
by  number 

13 

P < .01 

The  fact  that  S/N  ratio  was  not  a statistically  significant  factor 
Is  somewhat  surprising.  However,  whereas  the  range  of  values  for  number 
of  targets  and  visual  angle  was  extended  for  the  second  experiment 
(cf.  Table  3),  the  range  of  values  for  S/N  remained  the  same  as  In  the 
screening  experiment.  At  any  rate,  the  data  suggest  that  S/N  ratio  for 
the  values  considered  In  this  experiment  Is  not  an  Important  factor. 


A 
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Figures  4 and  5 show  percent  correct  (corrected  for  chance"*)  as  a 
function  of  number  of  targets  and  visual  angle,  respectively.  Both 
figures  indicate  an  increase  in  performance  between  the  lowest  and 
middle  value  of  these  two  Independent  variables.  However,  the  function 
appears  to  be  asymptotic  between  the  middle  and  upper  values  of  these 
factors . 


NUMBER  OF  TARGETS  VISUAL  ANGLE  (0)  IN 

MINUTES  OF  ARC 


FIGURE  4.  Percent  Correct  as  a 
Function  of  Number  of  Targets. 


FIGURE  5.  Percent  Correct  as  a 
Function  of  Visual  Angle  of  Targets. 


The  interaction  between  the  factors  contrast  and  configuration 
(Figure  6)  is  due  to  the  fact  that  linear  configuration  only  aided 
performance  when  the  targets  were  relatively  dark.  The  cause  of  this 
seems  to  be  that  dark  targets  were  about  the  same  luminance  as  trees  in 
the  terrain  scene.  When  the  dark  targets  were  in  a linear  configura- 
tion, they  were  easily  distinguishable  from  the  trees  because  of  their 
configuration.  When  these  same  targets  were  in  a random  configuration, 
the  subjects  apparently  confused  the  target  and  tree  configurations. 
Type  of  configuration  was  of  less  consequence  for  the  light  targets, 
because  they  were  not  easily  confused  with  trees  in  the  scene. 

The  same  type  of  explanation  seems  plausible  for  the  Interactions 
Illustrated  in  Figures  7 and  8.  Figure  7 shows  that  dark  targets  were 
most  easily  detected  for  the  four-target  groupings.  Since  most  of  the 
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configurations  of  the  trees  in  the  terrain  scene  were  for  groups  larger 
than  four,  apparently  neither  the  random  nor  linear  configurations  of 
dark  targets  were  easily  mistaken  for  tree  configurations  when  they 
contained  only  four  targets.  The  essentially  parallel  lines  in  the 
four-target  case  shown  in  Figure  8 support  this  contention. 

However,  the  seven-target  case  shows  a marked  interaction  of  con- 
figuration and  contrast  (Figure  8).  In  this  case,  the  dark  target 
conditions  show  a 35%  increase  in  performance  for  the  linear  as  opposed 
to  the  random  configuration.  Thus,  when  the  targets  are  configured  like 
the  trees  (l.e.,  dark  with  a large  number  of  targets),  having  a linear 
array  is  particularly  Important,  suggesting  that  configuration  was  a cue 
used  to  differentiate  target  and  tree  configurations.  This  possibility 
is  further  supported  by  the  subjects'  reports  that  the  random  target 
configurations  looked  like  clumps  of  trees  when  the  targets  were  dark. 

Configuration  is  not  plotted  as  a parameter  in  the  single-target 

case . 


80 


70 


60 


50 


40 


30 


20 


10 


A LINEAR  CONFIGURATION 
RANDOM  CONFIGURATION 


-L 


-L 


LIGHT  DARK 

(-27)  (-48) 

CONTRAST 


FIGURE  6.  Percent  Correct  as  a Function  of  FIGURE  7.  Percent  Correct  as  a Function  of 

Contrast  With  Configuration  as  a Parameter.  Number  of  Targets  With  Contrast  as  a Parameter. 
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FIGURE  8.  Percent  Correct  as  a Function  of  Contrast  With  Configuration 
as  a Parameter  Plotted  for  the  One-,  Four-,  and  Seven-Target  Situations. 


Regression  Analysis 


The  data  were  collapsed  over  subjects  and  a regression  analysis  was 
done  using  a "leaps  and  bounds"  regression  algorithm. This  algorithm 
chooses  the  best  models  for  each  r^'ibset  size  using  the  residual  sum  of 
squares  as  a Trlterion. 

Because  of  the  analysis  of  variance,  the  independent  variables  for 
possible  inclusion  in  the  model  were  modified.  The  last  two  levels  of 
the  visual  angle  and  number  of  targets  were  collapsed.  The  collapsed 
models  were  compared  to  noncollapsed  models  using  coefficient  of 
determination  (r2)  and  residual  plots ^ to  compare  the  two  classes  of 
models.  For  both  criteria,  the  collapsed  models  provided  a better  fit. 
The  collapsed  models  were  used  instead  of  other  possible  transformations 
because  the  data  suggested  that  performance  was  asymptotic  at  the  middle 
level  for  both  factors. 


*2g.  M.  Furnival  and  R.  W.  Wilson.  “Regression  by  Leaps  and  Bounds,”  Technonicirics, 
Vol.  16  (1974),  pp.  499-510. 
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The  values  for  contrast  were  treated  as  dummy  variables,  because 
the  two  contrast  conditions  were  essentially  discrete  values  which 
differed  qualitatively  as  well  as  quantitatively.  This  decision  was 
based  on  the  Interaction  patterns  in  Figures  6,  7,  and  8. 

Orthogonal  polynomial  coefficients  were  used  to  construct  a regres- 
sion model  which  was  sensitive  to  the  observed  interaction  effects. 
However,  the  model  accounted  for  very  little  additional  variance  and  would 
be  specific  to  the  particular  terrain  simulated  in  this  experiment. 

It  was  felt  that  a main  effects  model  would  be  more  robust  over  a variety 
of  terrain  conditions.  Models  generated  using  the  coefficients  are 
listed  in  Appendix  C. 

The  regression  models  chosen  using  the  "leaps  and  bounds"  algorithm 
are  reported  in  Table  6. 


TABLE  6.  Regression  Models. 


Subset 

size 

Model* 

2* 

1 

Y = 0.1055N  + 0.17 

0.32 

2 

Y = 0.1055N  + 0.0098V  + 0.165 

0.43 

3 

Y = 0.1055N  + 0.0098V  - 0.1975C  + 0.31 

0.47 

4 

Y = 0.1055N  + 0.0098V  - 0.1975C  + 0.19F  - 0.03 

0.50 

5 

Y = 0.1055N  + 0.0098V  - 0.1975C  + 0.19F  + 0.05  S/N  - 0.13 

0.52 

* Where 


Y = proportion  correct  decisions  corrected  for  chance 
N = number  of  targets 

V = visual  angle  of  target  in  minutes  of  arc 
C = contrast  of  target  to  background 

F = configuration  of  targets 
S/N  = signal- to-noise  ratio 

r2  = coefficient  of  determination  (this  is  a measure  of 
the  amount  of  variance  accounted  for) . 

Configuration  and  contrast  are  dummy  variables  with  the  fc’ lowing  values 
assigned  to  their  respective  levels:  1.0  to  linear  and  light  (-0.27); 
0.5  to  random  and  dark  (-0.48). 
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APPLICATIONS 


LIMITATIONS 

Regression  models  are  not  meant  to  show  functional  relationships 
among  the  variables  in  the  system  being  investigated,  but  rather  to  give 
a best  estimate  of  system  performance.^  The  four-term  model  is  the  most 
appropriate  model  for  predictive  purposes,  since  the  effect  of  noise  was 
not  found  to  be  statistically  reliable  in  the  analysis  of  variance. 

The  bounds  of  the  model  are  important  because  the  models  are  based 
on  the  observed  asymptotic  functions  for  the  visual  angle  and  number  of 
target  terms.  Past  research indicates  that  the  function  relating  the 
operator's  performance  to  the  target's  visual  angle  is  asymptotic,  as 
was  also  found  in  the  present  experiment.  However,  it  is  doubtful  that 
the  function  relating  performance  to  number  of  targets  is  asymptotic 
beyond  the  range  of  values  used  in  this  experiment. 

With  this  in  mind,  the  following  rules  should  be  observed  when 
using  the  regression  model.  First,  if  the  number  of  targets  or  visual 
angle  is  beyond  the  asymptotic  value  found  in  this  experiment,  use  the 
asymptotic  values  of  4 for  number  of  targets  and  27  for  visual  angle. 
Secondly,  if  the  values  of  these  two  factors  are  beyond  the  range  of  the 
values  used  in  the  experiment  (seven  for  number  of  targets,  47  minutes 
for  visual  angle)  then  the  predicted  percent  detections  will  be  under- 
estimated. A few  examples  of  possible  applications  for  the  regression 
model  should  help  in  understanding  possible  uses  of  the  model  in  opera- 
tional and  design  settings. 


OPERATIONAL  PREDICTIONS 

An  analyst  wishes  to  predict  the  probability  of  detecting,  P(D), 
a command  post  (6  targets)  at  a slant  range  of  2 nmi  from  an  attack 
aircraft.  He  computes  the  visual  angle  of  an  Individual  target  on  the 
display  to  be  about  15  minutes  of  arc  at  this  range  for  the  imaging 
device  used.  The  targets  are  believed  to  be  relatively  light  and  the 
configuration  is  random. 


'■^Nav;il  Weapons  tenter,  hmnc  IdeiiiifUatioii  on  Tckrisioii.  by  Ronald  Erickson  and  J.  1.. 
Ilcniini!way,  Cliitia  Lake.  (’A.  NWC.  September  1973.  (NWC  Tl’  .‘^025.  publication  UNCLASSIFII.D.) 
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P(D)  = (0.1055(4)  + 0,0098(15)  - 0.1975(1.0)  + 0.19(0.5)  - 0.03 
P(D)  = 0.44 

If  the  analyst  has  less  information,  he  can  use  one  of  the  models  with 
less  terms  listed  in  Table  7. 


PREDICTIONS  IN  TERMS  OF  DISPLAY  SIZE 

The  designer  is  interested  in  predicting  pilot  performance  under 
a variety  of  mission  and  design  conditions.  The  models  in  Table  6 
can  be  used  to  predict  performance  in  terms  of  display  size,  because  a 
direct  relationship  exists  between  display  size  and  visual  angle  of  the 
target  in  the  present  experiment. 

The  derivation  of  this  relationship  is  based  on  the  geometry  of  the 
viewing  situation.  Equation  4 relates  the  visual  angle  of  the  display 
(a)  to  the  viewing  distance  (D)  and  width  of  the  display  screen  (W) 
actually  used. 


/a\  _ W 

\2)  2D 


Equation  5 gives  the  equi\alent  display  size  (S)  if  the  distance  were 
kept  constant  (K)  and  a were  varied. 


S = 2K  tan( 


To  solve  for  S in  terms  of  visual  angle  of  the  target  (4))  , it  is  neces- 
sary to  know  the  size  of  the  target  (T)  Imaged  on  the  screen  as  shown  in 
Equation  6. 


Equation  7 solves  for  (p  and  substitutes  the  terms  S,  K,  and  W from 
Equation  5 into  Equation  6. 


= 2 tan 


-1/ST  \ 
\2KW/ 


1 
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For  the  present  experiment,  T equals  0.2  inch,  W equals  7 inches,  and 
the  pilot  is  assumed  to  be  25  inches  (K)  from  the  screen.  Equation  7 
. gives  a solution  of  (j)  in  terms  of  S or  i}>  ® 2 tan“l  (0.000562)  S.  This 

solution  is  in  degrees,  to  solve  for  4>  in  minutes  of  arc,  <}>  = 120  tan~i 
(0.000562). 

As  an  example,  the  designer  wishes  to  predict  probability  of  detec- 
tion as  a function  of  display  size  (for  a situation  that  is  comparable 
to  the  simulation  parameters  listed  in  Table  3) . He  is  also  Interested 
in  the  effect  of  target  contrast  and  the  one  target  versus  the  multi- 
target (4  to  7)  situation.  Assuming  that  the  pilot  sits  25  inches  from 
the  display,  120  tan~l  (0.000562)  (display  size)  can  be  used  instead  of 
' visual  angle  of  the  target  as  the  predictor  variable  in  the  regression 

equation  (in  this  case,  the  three-term  model  in  Table  6).  Figure  9 
shows  the  predicted  percent  correct  detections  in  this  case. 


•THE  MODEL  ASSUMES  PERFORMANCE  IS 
ASYMPTOTIC  BEYOND  A 7-INCH  DISPLAY 


FIGURE  9.  Predicted  Detection  Performance  Using  the  Regression  Model 
With  Display  Size,  Number  of  Targets,  and  Target  Contrast  as  Terms. 
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CORRECTIONS  FOR  VIBRATION 

Obviously,  the  reliability  of  the  predictions  from  the  regression 
models  depends  on  how  well  the  simulation  parameters  reflect  the  real- 
world  situation.  It  is  possible  to  correct  P(D)  for  variables  not 
included  in  the  model,  provided  the  correction  factor  is  known  and 
the  new  variables  do  not  interact  with  the  variables  Included  in  the 
model. 

As  an  example.  Figure  10  shows  curves  of  equal  decrement  of  perform- 
ance (percentage  correct)  plotted  against  acceleration  and  frequency  of 
vibratory  stimuli.^**  These  curves  can  only  be  used  as  approximate  cor- 
rection factors,  since  the  task  used  was  a number  reading  task.  However, 
there  is  little  reason  to  believe  that  vibration  interacts  with  the 
factors  included  in  the  regression  models  and  a ball  park  estimate  of 
the  effects  of  vibration  is  advisable  in  making  any  realistic  estimate 
of  system  performance  for  Naval  aircraft. 


DISCUSSION 


GENERAL  FINDINGS 

The  findings  suggest  that  the  factors  which  influence  detection 
performance  the  most  are  operational  in  nature.  That  is,  they  are 
variables  that  are  not  under  the  direct  control  of  the  designer  but 
change  as  a function  of  parameters  of  the  operator's  mission.  Specifi- 
cally, the  size  of  the  display  necessary  for  detection  depends  on  how 
many  targets  are  being  looked  for,  target  contrast  and  configuration. 

One  reason  that  some  of  the  factors  were  not  found  to  be  more 
important  may  relate  to  the  values  of  the  factors  used  in  the  experi- 
ments. For  example,  Levine  and  Youngling  found  that  stabilized  images 
particularly  helped  performance  at  airspeeds  higher  than  those  simulated 
in  the  present  experiment.**  Therefore,  mode  of  presentation  may  have 
been  a more  important  factor  at  higher  simulated  airspeeds.  Similarly, 
if  the  sampling  rate  of  the  auxilary  tasks  in  the  number  of  tasks  con- 
dition had  been  increased,  airspeed  and  mode  of  presentation  (as  well  as 
number  of  tasks)  may  have  shown  a concomitant  Increase  in  relative 
Importance.  The  values  chosen  for  the  experiments  seemed  reasonable  in 
view  of  the  operational  requirement  of  modern  Naval  aircraft. 


S.  J.  Morrissey  and  A.  C.  Bittner.  “Effects  of  Vibrations  on  Humans:  Performance 
Decrements  and  Limits.”  in  Proceedings  of  the  Human  Factors  Society,  1977,  pp.  6K-72. 

McDonnell  Douglas  Astronautics  Company,  East.  Real  Time  Target  Acquisition  with 
Moving  and  Stabilized  Image  Di^lays,  by  S.  H.  Levine  and  E.  W.  Youngling.  St.  Louis.  MO, 
MDC,  1973.  (MDC  E0769,  publication  UNCLASSIFIED.) 
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FREQUENCY,  Hz 


FIGURE  10.  Isodecrement  Curves  (Percent  Correct)  Plotted  Against  Acceleration 
and  Frequency  of  Vibratory  Stimuli  (Morrissey  and  Bittner,  1977). 


Another  reason  some  factors  were  not  shown  to  be  important  may  be 
the  specific  task  used  in  the  experimental  situations.  Signal-to-noise 
ratio  and  resolution,  for  the  values  in  the  present  experiment,  affect 
detail  rather  than  overall  shape  of  both  the  target  and  background. 
Detection  depends  on  the  general  energy  level  or  shape  of  the  targets, 
whereas  recognition  and  classification  depend  on  specific  features.^® 
Thus,  S/N  ratio  and  resolution  could  be  expected  to  have  a greater 
Impact  on  recognition  and  classification  performance  than  on  detection 
performance.  Also,  the  subject's  cognitive  uncertainty  was  unrelated 
to  detection  performance,  although  past  research  indicates  it  does  affect 
classification  performance.'® 


(i.  Ganioi.  Tlw  Praicssin^  of  Infonnation  mul  Sfnwiiirc.  Potomac,  MD,  Lawrence 
Irlhamn  Associates,  I't74. 
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The  regression  model  that  was  generated  (4-term  model)  only  accounts 
for  50%  of  the  variance.  This  means  that  uncontrolled  factors  had  an 
important  effect  on  detection  performance.  The  most  obvious  variable 
that  was  not  measured  was  the  terrain  type.  The  same  four  scenes  were 
repeated  in  each  experimental  cell.  However,  the  placement  of  the 
targets  in  the  terrain  was  randomized  with  the  provision  that  they  were 
placed  either  to  the  right  or  left  and  upper  or  lower  quadrant  equally 
often  for  each  experimental  condition.  This  placement  was  approximate 
and  it  seems  likely  that  the  specific  placements  of  the  target  on  the 
terrain  may  have  contributed  to  experimental  variance . 

Research  by  Scanlan^^  (1977)  indicates  that  target  placement  is  an 
important  variable  for  target  acquisition  performance.  Quantification 
of  target  placement  variables  and  terrain  differences  using  either 
regression  or  multidimensional  analysis  is  a necessary  first  step  in 
developing  a detection  model  that  Includes  factors  related  to  the 
physical  parameters  of  the  terrain. 

Subject  differences  accounted  for  important  variation  in  both 
experiments.  Order  effects  accounted  for  0.60  of  the  subject  variation 
in  the  screening  experiment,  suggesting  that  learning  variables,  or 
interaction  between  conditions  might  influence  performance.  An  analysis 
of  subject  difference  was  conducted  using  statistics  developed  from 
signal  detection  theory. 


SUBJECT  DIFFERENCES  AND  SIGNAL  DETECTION  THEORY 

Signal  detection  theory^®  requires  that  the  subject  gather  evidence 
concerning  two  alternatives  and  compute  the  odds  concerning  their  relative 
veracity.  6 is  the  criterion  the  subject  uses  to  make  his  decision.  If 
the  odds  are  above  (or  equal  to)  S he  chooses  one  alternative  and  if 
they  are  below  3 he  chooses  the  other  alternative. 

For  psychophysical  experiments,  the  alternatives  are  considered  to 
be,  (1)  noise  and  (2)  signal  with  noise. In  the  present  experimental 
paradigm,  these  alternatives  were  represented  by  trials  with  (1)  terrain 
scenes  alone,  and  (2)  the  scene  with  a target  located  somewhere  in  it. 
d'  is  a measure  of  the  subject's  ability  to  differentiate  between  the  two 
alternatives  (l.e.,  his  sensory  sensitivity)  and  is  independent  of  the 
subject's  6 (i.e.,  his  decision  strategy). 


L.  W.  Scunlan.  “Target  Acquisition  in  Realistic  Terrain,"  in  Proceedings  of  the  Hiiniaii 
l-aciors  Society,  1^77,  p.  249. 

M.  Green  and  J.  A.  Swets.  Signal  netection  Theory  and  Psych(>physics.  New  York, 
John  Wiley  and  Sons,  1964. 

28 


1 


NWC  TP  6006 


Performance  depends  on  both  d'  and  6.  There  is  an  optimal  S 
depending  on  the  prior  probability  of  the  targets  and  the  utility  of 
choosing  one  alternative  over  the  other.  In  the  present  experiments, 
the  subjects  were  informed  that  both  the  prior  probability  and  utility 
of  saying  "yes"  or  "no"  (as  to  whether  a terrain  scene  contained  a 
target)  were  equal.  In  this  particular  case,  the  optimal  3 is  one 
(odds  are  1 to  1)  and  if  the  subject's  decision  strategy  favors  either 
alternative  his  percent  detection  score  will  be  affected  adversely. 

An  estimate  of  the  subjects'  d'  and  3 were  made  for  both  experi- 
ments and  used  to  analyze  subject  differences  (Table  7). 


TABLE  7.  Signal  Detection  Analysis  of  Subject  Differences. 


Subject 

Probability  of 
detection,  P(D)‘^ 

3 

A.  Experiment  1 

1 

.66 

2.16 

2.78 

2 

.70 

2.08  " 

0.84 

3 

.80 

2.78 

2.83 

4 

.73 

2.38 

2.94 

5 

.62 

1.90 

2.11 

6 

.54 

1.56 

1.86 

B.  Experiment  2 

1 

.37  (6)* 

1.15  (6)* 

2.13  (3)* 

2 

.50  (5) 

1.70  (3) 

3.31  (5) 

3 

.53  (3) 

1.82  (1) 

3.33  (6) 

4 

.54  (2) 

1.73  (2) 

2.54  (4) 

5 

.48  (4) 

1.40  (5) 

2.00  (2) 

6 

.54  (1) 

1.47  (4) 

1.65  (1) 

^ Corrected  for  chance. 

^ Numbers  in  parentheses  Indicate  rank. 


In  the  first  experiment,  most  of  the  subjects  favored  missing 
targets  rather  than  committing  errors  of  false  detection,  which  was 
indicated  by  a 3 of  2 or  3.  Subject  2 had  a 3 close  to  1 (nearly 
optimal).  His  measured  sensory  sensitivity  (d')  was  lower  than  that 
of  subject  1 but  his  probability  of  detection,  P(D) , was  better.  In 
other  words,  he  apparently  saw*  fewer  targets  than  subject  1,  but  was 
able  to  maximize  his  P(D)  by  using  an  optimal  decision  strategy. 


* III  the  sense  that  clutter  and  targets  were  not  as  easily  differentiated. 
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The  effect  of  decision  strategy  is  more  marked  in  the  second 
experiment.  Subjects  were  even  more  disposed  toward  missing  targets 
as  opposed  to  making  false  detections.  Subject  1 had  the  best  P(D) 
score  (although  only  minimally  better  than  subject  3 or  4) , but  his  d' 
score  ranked  fourth,  whereas  his  3 was  the  closest  to  optimal.  This 
means  that  in  relation  to  the  other  subjects  his  ability  to  detect  the 
target  was  poor  but  his  decision  strategy  was  good.  His  d'  score  was 
23%  less  than  that  of  subject  3,  but  his  P(D)  was  slightly  better. 

In  general,  the  subject's  rank  for  P(D)  and  d'  are  not  well  correlated, 
suggesting  that  the  subject's  P(D)  scores  are  due  to  decision  strategy 
as  well  as  sensory  sensitivity. 


These  results  are  more  suggestive  than  conclusive,  since  in  terms 
of  absolute  magnitude  the  effects  are  small.  However,  the  experimental 
situation  did  not  manipulate  factors  which  traditionally*®  affect 
decision  strategy  and  the  optimal  strategy  was  emphasized  to  the 
subjects.  The  effect  of  decision  strategy  on  detection  performance 
should  be  investigated  under  conditions  more  like  actual  operational 
settings  (i.e.,  different  probabilities  and  utilities  for  events  to 
occur).  If  P(D)  is  a result  of  3 as  well  as  d' , different  design 
strategies  are  called  for.  For  example,  P(D)  might  improve  more  by 
giving  the  pilot  realistic  prior  probabilities  for  target  occurrence 
rather  than  increasing  the  display  size. 


I \ i 


; t : 
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SUMMARY 


1.  The  physical  size  of  the  display  (when  MTF  and  visual  angle 
were  controlled)  proved  to  be  a relatively  unimportant  factor  in  the 
screening  experiment. 

2.  Most  of  the  factors  that  were  Included  in  the  regression  model 
(number  of  targets,  target  contrast,  and  target  configuration)  were 
operational  factors  rather  than  design  parameters. 

3.  Derivation  of  the  relationship  between  the  visual  angle  of  the 
target  and  display  size  allowed  performance  to  be  predicted  in  terms  of 
display  size. 

4.  Configuration  cues  such  as  linearity  and  number  of  targets 
interacted  with  target  contrast,  suggesting  a complex  relationship 
between  these  factors  and  terrain  parameters. 

5.  Most  subjects  were  non-optimal  in  their  decision  strategy  (6) 
(l.e.,  they  were  willing  to  miss  many  potential  targets  rather  than  make 
false  detections) . Differences  in  decision  strategies  (3)  were  as 
Important  as  differences  In  sensory  processing  abilities  (d')  in 
explaining  performance  differences  among  subjects. 
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Appendix  A 

ENGINEER’S  REPORT  ON  DISPLAY  PERFORMANCE  MATCHING 

by  Richard  Hensley 


The  display  size  experiment  conducted  by  the  Human  Factors  Branch, 
Code  3175,  required  that  two  displays  of  different  screen  size  have 
equal  performance.  The  following  is  a discussion  of  how  the  performance 
of  two  displays  was  measured  and  matched. 

The  two  displays  chosen  for  the  experiment  were  a 9-inch  Conrac 
and  a 3-lnch  3Q.  The  measure  of  performance  used  against  the  two 
displays  was  the  Modulation  Transfer  Function  (MTF) . MTF  is  the  Fourier 
transform  of  the  display's  rise  time  response  from  a black  to  white  edge 
transition.  With  matched  MTF,  the  displays  will  respond  equally  to  a 
given  input . 

The  measurement  of  MTF  was  accomplished  by  utilizing  a scanning 
microphotometer.  The  microphotometer  scanned  the  black  to  white  edge 
transition,  measuring  light  amplitude  as  a function  of  distance  across 
the  scene.  Figure  A-1  is  a block  diagram  of  the  measurement  process. 
Table  A-1  is  an  equipment  list. 


FIGURE  A-1 . MTF  Measurement  Block  Diagram. 
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The  data  from  the  scanning  photometer  were  then  processed  by  a i 
Wang  2200  programmed  for  doing  Fourier  transforms.  Figure  A-2  shows  | 
the  MTF  curves  for  both  displays.  j 

The  Conrac  display  had  significantly  better  performance  than  the  | 
3Q  display.  The  Conrac  display  was  3 dB  (0.707  amplitude)  down  at  j 
0.45  cycles/mm.  The  3Q  display  was  3 dB  down  at  0.17  cycles/nm.  The  j 
Conrac  performance  must  be  reduced  to  match  that  of  the  3Q.  j 

i 

The  performance  of  the  Conrac  was  reduced  by  adjusting  the  brightness  ! 
and  contrast  controls.  However,  brightness  and  contrast  could  not  reduce  • I 
the  performance  enough  to  match  the  3Q.  The  brightness  and  contrast  1 
controls  were  returned  to  a setting  near  that  of  the  3Q.  A single  pole,  j 
RC  low-pass  filter  having  a time  constant,  T,  of  0.28  Msec  was  then  "j 
inserted  at  the  input  of  the  Conrac  display.  Figure  A-3  is  the  measured  j 
frequency  response  of  this  filter.  ! 

! 

With  filter  in  place,  MTF  data  were  again  taken  and  processed.  j 
Figure  A-4  shows  the  adjusted  Conrac  and  3Q  MTF  curves.  Tlie  Conrac  I 
3-dB  point  is  now  0.180  cycles/mm.  Tills  nearly  matches  the  I 
0.170  cycles/mm  of  the  3Q  display.  i 


• ; 


FIGURE  A-3.  In-Line  Filter  Schematic  and  Bandwidth. 
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Appendix  B 


EXPERIMENTAL  DESIGN  AND  RESULTS 

Experimental  conditions  for  the 

partial  factorial 

experiment 

Code 

Factor 

High  value 

Low  value 

A 

Display  size 

9 

2 

B 

Visual  angle 

22  min 

11  min 

C 

Airspeed 

460  knots 

259  knots 

D 

Mode  of  presentation 

Snow  plow 

Stills 

E 

Uncertainty 

1 bit 

2 bits 

F 

Contrast 

-.48 

-.27 

G 

S/N 

28  dB 

12  dB 

H 

Resolution 

300  lines 

175  lines 

I 

No . of  tasks 

One 

Three 

J 

No.  of  targets 

Six 

Three 

Experimental  (runs)  conditions:  Letter  means  that  high  value  of 
that  factor  is  used  for  that  condition,  low  value  assumed  otherwise. 


Block  I:  Basic 

Block  II:  Fold  over 

Cond 

High  value  factors 

Cond 

High  value  factors 

1 

ABCDEFGHIJ 

17 

All  low  values 

2 

ABDH 

18 

CEFGIJ 

3 

ACEFH 

19 

BDGIJ 

4 

AGHIJ 

20 

BCDEF 

5 

ACDGJ 

21 

BEFIH 

6 

ABCI 

22 

DEFGHJ 

7 

ABEFGJ 

23 

CDHI 

8 

ACEFI 

24 

BCGHJ 

9 

BECGH 

25 

ADEIJ 

10 

BFHIJ 

26 

ACEGD 

11 

DFGH 

27 

ABCEIJ 

12 

CDEHIJ 

28 

ABFG 

13 

EJ 

29 

ABCDFGHI 

14 

EFGI 

30 

ABCDHJ 

15 

BDEGI 

31 

ACFHJ 

16 

BCDFG 

32 

AEHl 
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ANOVA  TABLE  (Display  Size:  Experiment  2) . 


Error  term  is  the  next  higher  order  interaction  which  includes 
only  the  factor(s)  to  be  tested  and  subjects  (F) . 

F Score  only  given  for  statistically  significant  fa'tors  and 
interactions . 


Factor* 

MSa 

DF* 

F Score‘s 

0(d 

Factor* 

MSa 

DF* 

F Scored 

Q(d 

A 

6.52 

1 

0.01 

ABF 

0.58 

5 

• • • 

B 

6.125 

1 

0.01 

ACF 

0.39 

10 

C 

30.19 

2 

47 

0.01 

BCF 

0.48 

10 

D 

2.24 

2 

• « 

• • • 

ADF 

0.46 

10 

E 

11.31 

2 

40 

0.01 

BDF 

0.38 

10 

F 

1.66 

5 

CDF 

0.53 

20 

AB 

6.52 

1 

11 

0.05 

1.64 

10 

• . . 

AC 

0.04 

2 

0.35 

10 

BC 

3.97 

2 

8 

0.01 

0.852 

20 

AD 

0.10 

2 

0.800 

20 

BC 

0.197 

2 

0.37 

4 

. 

BD 

0.199 

2 

ABCE 

1.28 

4 

CD 

0.428 

4 

0.30 

4 

AE 

0.168 

2 

• • • 

0.64 

8 

BE 

0.81 

2 

1.54 

8 

CE 

0.129 

4 

ABCF 

0.283 

10 

DE 

0.235 

4 

0.52 

10 

AF 

0.183 

5 

ACDF 

0.55 

20 

BF 

0.284 

5 

0.55 

20 

CF 

0.64 

10 

ABEF 

0.58 

10 

• • • 

DF 

0.54 

10 

0.46 

20 

EF 

0.28 

10 

BCEF 

0.65 

20 

ABC 

3.65 

13 

0.01 

0.86 

20 

ABD 

0.14 

0.62 

20 

ACD 

0.29 

0.68 

40 

BCD 

0.27 

ABCDE 

1.94 

8 

ABE 

0.29 

• • • 

ABCDF 

0.52 

20 

ACE 

0.39 

ABCEF 

0.49 

20 

BCE 

1.37 

. • • 

ABDEF 

0.81 

20 

ADE 

0.467 

ACDEF 

0.77 

40 

BDE 

0.21 

BCDEF 

0.38 

40 

CDE 

1.43 

8 

ABCDEF 

0.87 

40 

1 



* Factors:  A = Alignment;  B = Contrast;  C = Number  of  targets; 

D = Signal-to-noise  ratio;  E » Target  visual  angle;  F = Subject. 

•3  MS  = mean  squares;  * DF  = degrees  of  freedom;  F Score  = MS  factor 
r MS  error  term;  a = significance  level. 
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Appendix  C 

REGRESSION  MODELS  USING  ORTHOGONAL 
POLYNOMIAL  COEFFICIENTS 


Variable 

Symbol 

B weight 

Bi 

Coefficient 

coding 

Level  1 

Code 

Level  2 

Code 

Level  3 

Code 

No.  of  targets 

N 

0.1676 

Bl 

1 

-1 

4 

0 

7 

1 

(No.  of 
targets) ^ 

n2 

-0.0435 

B2 

1 

1 

4 

-2 

7 

1 

Contrast 

C 

-0.0494 

B3 

-0.48 

-1 

-0.27 

n 

... 

. • • 

Configuration 

F 

0.048 

B4 

random 

-1 

linear 

H 

. . . 

. . . 

Signal-noise 

S/N 

0.051 

B5 

13 

-1 

17 

0 

28 

1 

Contrast  by 
configuration 

CF 

-0.0495 

B6 

. . . 

■ 

No.  targets 
by  contrast 

NC 

-0.0397 

B7 

• • • 

H 

Contrast  by 
configuration 
by  No.  of 
targets 

NCF 

-0.0408 

B8 

1 

Visual  angle 

V 

0.195 

B9 

7 

-1 

27 

0 

47 

1 

Intercept 

A 

1.98 

Bo 

. . . 

• • • 

Subset 

size 

Best  fit 

; models 

2 

r 

1 

Y 

= 

BjN 

+ 

«0 

26 

2 

Y 

= 

BjN 

+ 

BgV 

+ 

38 

3 

Y 

= 

B^N 

+ 

B9V 

+ 

2 

B.N 
^ 2 

+ 

»0 

43 

4 

Y 

B^N 

+ 

B9V 

+ 

B.N 
^ 2 

+ 

BbCF 

+ 

»0 

47 

5 

Y 

= 

Bj^N 

+ 

B9V 

+ 

B.N 
^ 2 

+ 

BbCF 

+ 

B3C 

+ 

®0 

50 

6 

Y 

= 

B^N 

+ 

B9V 

+ 

B„N 
^ 2 

+ 

BbCF 

+ 

B^C 

+ 

+ 

«0 

54 

7 

Y 

Bj^N 

+ 

B9V 

+ 

B,N 
^ 2 

+ 

B,CF 

D 

+ 

B3C 

+ 

B4F 

+ 

B3S/N  + 

»0 

56 

8 

Y 
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INSTRUCTIONS:  EXPERIMENT  1 


This  Is  a simulated  reconnaissance  mission.  Your  task  will  be  to 
detect  possible  targets:  missiles,  tanks,  houses,  or  trucks.  These 
targets  are  displayed  on  the  table  In  front  of  you  and  will  either  be 
fairly  light  (the  tan  ones)  or  dark  (dark  gray  ones) . The  targets  will 
be  displayed  In  groups  of  from  three  to  six.  You  don't  have  to  Identify 
what  the  targets  are.  Rather  your  task  will  be  to  detect  possible  target 
configurations.  You  will  be  told  what  targets  are  possible  before  each 
run. 

We  will  move  over  to  the  simulate'  display.  This  Is  the  response 
box,  hold  It  so  It  Is  comfortable.  The  TV  monitor  I am  pointing  at  Is 
the  display  the  simulated  terrain  will  be  Imaged  on.  First  you  will  see 
a card  with  a number  on  It.  A few  seconds  after  this  an  Image  of  the 
terrain  will  replace  the  card. 

Next,  a buzzer  will  sound  Initiating  the  first  trial.  This  will 
signal  you  to  look  at  the  terrain  carefully  for  a possible  target. 

When  the  second  buzzer  sounds,  you  will  make  a forced  choice  by 
pressing  the  "yes"  button.  If  there  had  been  a target  since  the  previous 
buzzer;  otherwise  press  the  "no"  button.  There  will  be  nine  buzzers  for 
each  experimental  run.  You  will  ignore  the  first  buzzer  and  respond  to 
each  of  the  next  eight  buzzers.  Respond  as  quickly  as  possible  to  each 
buzzer  since  you  will  be  timed. 

Again,  do  not  respond  to  the  first  buzzer,  but  for  each  successive 
buzzer  decide  whether  you  detected  a possible  target  configuration  since 
the  previous  buzzer  and  respond  as  quickly  as  you  can. 

There  are  two  possible  modes  of  presentation.  One  mode  will  consist 
of  still  images  of  the  terrain  which  will  replace  each  other  right  after 
the  buzzer  sounds.  The  target  can  be  any  place  on  the  Image  for  this 
mode.  In  the  other  mode,  the  terrain  will  be  moving  through  your  tele- 
vision screen.  Immediately  after  the  buzzer  ending  one  trial,  targets 
for  the  next  trial  will  come  Into  your  field  of  view  for  trials  in  which 
the  targets  are  present.  In  this  mode,  the  best  strategy  Is  to  fixate 
on  a place  near  the  top  of  the  screen  and  search  the  horizontal  plane 
because  the  targets  will  pass  through  this  point.  If  they  are  present 
for  that  trial.  Approximately  half  the  trials  will  contain  targets. 

That  is,  there  is  a 50-50  chance  that  a target  will  be  present  for  a 
particular  trial.  Missing  targets  on  trials  where  there  was  one  will 
count  against  you  as  much  as  saying  there  was  a target  for  trials  In 
which  there  was  no  target. 
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There  will  be  five  gauges  on  each  screen.  A gauge  Indicator  will 
move  back  and  forth.  If  the  Indicator  stays  In  tolerance.  It  will  stay 
In  the  light  portion  of  the  display.  When  It  goes  out  of  the  light 
portion  and  turns  dark,  it  Is  out  of  tolerance.  You  will  indicate 
whether  It  Is  out  of  tolerance  by  noting  whether  display  A or  B Is  out 
of  tolerance,  and  also  the  direction  left  or  right.  For  example  you 
might  say  "screen  A,  left"  indicating  that  the  display  marked  A Is  out 
of  tolerance  to  the  left. 

Are  there  any  questions?  We  will  run  through  a number  of  practice 
trials,  if  you  have  any  questions  be  sure  to  clear  them  up  during  the 
practice  session. 
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INSTRUCTIONS:  EXPERIMENT  2 


The  experiment  will  consist  of  a simulated  reconnaissance  mission. 

Your  mission  will  be  to  detect  possible  target  configurations.  It  Is 
not  necessary  to  Identify  possible  targets.  Rather,  you  will  report 
possible  target  configurations.  For  the  practice  session,  tanks,  houses, 
or  trucks  will  be  possible  target  configurations.  In  the  experimental 
session,  only  tanks  or  trucks  are  possible  targets.  Over  here  on  the 
terrain  model  are  examples  of  possible  target  configurations.  The 
configurations  will  vary  from  one  to  seven  targets.  They  will  be  in  a 
straight  line  or  randomly  dispersed  and  will  be  either  light  or  dark. 

Let's  go  over  to  the  mock  display  and  sit  in  front  of  the  experimental 
monitor.  The  headrest  will  keep  you  the  proper  distance  from  the  dis- 
play, keep  your  forehead  flush  on  the  rest.  This  is  your  response  box. 

The  monitor  In  front  of  you  will  simulate  a ground-stabilized  Imaging 
device  found  in  aircraft.  You  will  see  a still  picture  of  terrain 
lasting  5 seconds.  These  Images  of  the  terrain  will  be  replaced  by 
pictures  of  the  terrain  every  5 seconds.  A tone  will  signal  the  end  of 
a scene's  duration.  When  you  hear  the  tone  you  must  make  a decision 
whether  a possible  target  configuration  was  present  or  absent  in  the 
terrain  scene  just  ending.  If  you  decide  that  a target  configuration 
was  present  press  the  "yes"  button,  and  If  you  decide  that  the  scene 
did  not  contain  a configuration  press  the  "no"  button. 

Remember  that,  under  some  conditions,  it  will  be  very  difficult  to 
decide  whether  a target  Is  present  or  whether  what  you  are  looking  at  Is 
terrain  clutter.  You  must  use  your  best  Judgment  and  respond  to  the 
tone  at  the  end  of  each  scene.  Responding  no  when  targets  are  present 
will  influence  your  score  as  much  as  responding  yes  to  scenes  In  which 
targets  are  absent.  Over  the  whole  experiment  half  the  scenes  you  will 
see  will  have  targets  and  half  will  not.  The  time  between  tones  (that 
is,  the  duration  of  one  scene)  will  constitute  one  trial.  A trial  will 
take  5 seconds.  (Show  Briefing  card.) 

A target  run  will  consist  of  16  trials.  There  will  be  27  runs.  At 
the  beginning  of  each  run  a card  will  indicate  the  run  number  and  inform 
you  the  number  of  targets  to  look  for  in  the  upcoming  run. 

Every  three  runs,  the  word  change  will  be  added  to  the  card  starting 
the  run.  The  experimenter  will  give  you  about  a minute  rest.  After 
nine  runs  you  will  get  a 5-minute  break.  An  experimental  session  will 
take  approximately  an  hour . 

For  the  rest  of  the  morning  session,  you  will  run  practice  trials. 

I will  show  you  the  targets  for  the  first  block  of  runs,  then  you  will 
run  through  the  block  yourself  and  I will  give  you  feedback  concerning 
performance.  You  will  then  run  two  additional  blocks  of  runs  under  more 
difficult  conditions.  During  the  practice  session,  you  may  interrupt 
anytime  to  ask  questions.  Do  you  have  any  questions  about  the  instructions? 
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